My study note

Equations

Equations

Equations Equations Equations

Network Socket Programming

Hands-On Network Programming with C Book by Lewis Van Winkle


Hands-On Network Programming with C

Github

Hands-On-Network-Programming-with-C-master

Reference Book

Data & Computer Communications Book by William Stallings


Data & Computer Communications

Lecture Series for Computer Network


TCP Dumps Examples

Additional Reading




Quick Reference

TCP/IP Model

Four layers, not seven. This is the stack actually implemented in operating-system kernels, and the one packet captures show on the wire.

Application: the protocol your code reads and writes Application HTTP, DNS, SSH, TLS, gRPC PDU message Transport: ports and either reliable TCP streams or unreliable UDP datagrams Transport TCP, UDP, QUIC, ports PDU segment Internet: addressing and routing across networks Internet IPv4, IPv6, ICMP, routing PDU packet Link: frames on a single physical or wireless segment; the NIC driver lives here Link Ethernet, Wi-Fi, ARP, NIC driver PDU frame

Send: each layer adds its header on the way down. Receive: the reverse, peel a header at each layer going up.

LayerJobExamples
ApplicationApp protocol; payload your code reads and writesHTTP, DNS, SSH, TLS, gRPC
TransportPort-to-port delivery; reliability, ordering, congestion controlTCP, UDP, QUIC
InternetHost-to-host routing across networksIPv4, IPv6, ICMP
LinkFrames on a single physical or wireless segmentEthernet, Wi-Fi, ARP, the NIC driver

The older 7-layer OSI model splits Application into Application/Presentation/Session, but nobody implements those as distinct layers. TLS is part of the app, framing is in the link, sessions belong to the protocol.


TCP vs UDP

TCPUDP
ConnectionYes (3-way handshake)No
ReliabilityGuaranteed deliveryBest-effort
OrderingIn-orderNone
Header20–60 bytes8 bytes
Use casesHTTP, SSH, emailDNS, video, VoIP

Common Ports

ServicePortServicePort
HTTP80HTTPS443
SSH22Telnet23
FTP21SMTP25
DNS53DHCP67/68
POP3110IMAP143

IPv4 vs IPv6

IPv4IPv6
Address size32 bits128 bits
Notation192.168.1.12001:db8::1
Header20–60 bytes40 bytes (fixed)
Address space~4.3 × 109~3.4 × 1038


iperf3: measuring throughput

iperf3 measures end-to-end throughput between two hosts. One side runs as a server, the other as a client. What you get back is the bandwidth the network actually delivers, not the line rate printed on the cable.

CommandWhat it does
iperf3 -sRun a server on default port 5201
iperf3 -c host -t 3030-second TCP test against host
iperf3 -c host -P 44 parallel streams; useful on high-BDP links
iperf3 -c host -u -b 100MUDP at 100 Mbit/s; reports jitter and loss
iperf3 -c host -RReverse mode: server sends, client receives
iperf3 -c host -w 4MOverride TCP window size (skip autotune)
iperf3 -c host -JEmit JSON for scripts and dashboards

A single TCP stream is often capped by the bandwidth-delay product and the receive window, not by the NIC. Use -P to find the link ceiling. For jitter and packet loss, use UDP; TCP hides loss inside retransmits.

Reading the output
ColumnMeaning
BitrateThroughput over the interval
RetrTCP retransmissions; non-zero means loss or queue overflow
CwndSender's congestion window (TCP)
JitterVariation in inter-packet arrival time (UDP)
Lost/TotalUDP packet loss ratio


Linux netdev: the network driver layer

Every NIC in Linux shows up as a net_device (eth0, wlan0, lo). The driver registers it with the kernel; the stack talks to it through a small set of callbacks called net_device_ops. Packets travel through as sk_buff structs (skbs), with the headers walked using offset pointers.

TX path (send) RX path (receive) User calls write(); the kernel copies bytes into the socket send buffer socket write() TCP segments the data and IP prepends a header; the result is an sk_buff TCP / IP build skb Queuing discipline: pfifo_fast, fq, fq_codel; can shape, prioritize, or drop qdisc (per-device queue) Driver callback: maps the skb, writes a TX descriptor, rings the NIC doorbell driver ndo_start_xmit Fixed-size DMA descriptor ring shared between driver and NIC TX ring (DMA) User calls read(); kernel returns bytes from the socket receive buffer socket read() TCP reassembles and acks; IP strips its header; payload queues on the socket TCP / IP to socket Entry point into the protocol stack from the driver netif_receive_skb NAPI poll runs in softirq, drains the ring in batches, then re-enables IRQ NAPI poll (softirq) NIC DMAs frames into RX descriptors and raises an interrupt RX ring + IRQ NIC hardware (PHY, MAC, DMA engine) physical link (Ethernet, fiber, Wi-Fi)
Key objects in the driver
SymbolWhat it is
net_deviceKernel object per interface (eth0, wlan0, lo); created with alloc_etherdev(), registered with register_netdev()
net_device_opsVtable of driver callbacks: ndo_open, ndo_stop, ndo_start_xmit, ndo_get_stats64, etc.
sk_buff (skb)Packet container with payload, header offsets, length, device pointer, and timestamps
napi_structNAPI context; napi_schedule() from the IRQ, napi_complete_done() when the ring is drained
qdiscQueueing discipline between IP and the driver: pfifo_fast, fq, fq_codel
DMA ringFixed array of descriptors; head/tail pointers cycle as packets are produced and consumed
Skeleton of a netdev driver
static const struct net_device_ops my_ops = {
    .ndo_open       = my_open,        /* ifconfig up: allocate rings, request IRQ */
    .ndo_stop       = my_stop,        /* ifconfig down: free rings, drop IRQ */
    .ndo_start_xmit = my_xmit,        /* TX one skb: map DMA, write descriptor */
    .ndo_get_stats64 = my_stats,      /* 64-bit byte/packet counters */
};

static int my_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
    struct net_device *ndev = alloc_etherdev(sizeof(struct my_priv));
    ndev->netdev_ops = &my_ops;
    netif_napi_add(ndev, &priv->napi, my_poll, NAPI_POLL_WEIGHT);
    return register_netdev(ndev);
}
Tools for poking at a NIC
CommandWhat it shows
ip linkList interfaces, link state, MAC, MTU
ip -s link show eth0RX/TX byte and error counters
ethtool eth0Link speed, duplex, supported modes
ethtool -i eth0Driver name, version, firmware
ethtool -S eth0Per-driver stats (queue counters, NAPI polls, drops)
ethtool -g eth0Ring sizes, current and maximum
ethtool -k eth0Offloads: TSO, GRO, checksum, RSS
tcpdump -i eth0Tap above the driver via libpcap
ss -tiPer-socket TCP info: cwnd, rtt, retrans

Why NAPI exists: at 10 Gbit/s a small-packet workload can push over a million packets per second. One IRQ per packet would melt a CPU. The driver raises a single interrupt, then a softirq drains the ring in a poll loop until it is empty or the budget is hit. Interrupts get re-enabled only when there is nothing left to do.



TCP/IP over SPI: Linux host + Wi-Fi co-processor

A Linux board without built-in Wi-Fi can still expose a normal wlan0 to user space. The trick: an MCU on the other end of an SPI bus carries the radio, and a small kernel driver makes the SPI link look like a regular netdev.

Linux host Wi-Fi MCU (e.g., ESP32) User app calls socket() / write() / read(); the kernel does the rest app: socket write() / read() TCP, IP, qdisc; produces an sk_buff destined for wlan0 TCP / IP / qdisc SPI netdev driver: ndo_start_xmit packs the skb into an SPI frame SPI netdev driver SoC SPI master with DMA; clocks the bus at 10-40 MHz SPI master + DMA SPI slave on the MCU; DMA into a ring of receive buffers SPI slave + DMA Framing, control channel, optional TCP/IP on the MCU co-processor firmware Wi-Fi MAC: builds 802.11 frames, handles ACK and retransmit 802.11 MAC RF baseband and PA; modulates onto 2.4 / 5 GHz RF PHY + antenna SPI bus: SCLK, MOSI, MISO, /CS; INT wakes the host on RX SPI bus SCLK · MOSI · MISO · /CS INT (MCU → host) to user space: wlan0 looks like any other netdev (sockets, iperf3, tcpdump all work) blue: TX (host → air) green: RX (air → host) orange: INT line
Where does the IP stack live?
ModeIP / TCPHost CPUTrade-off
Thin co-processor (mac80211 style)Linux kernelHigher: every packet crosses the SPI bus and the stackFull kernel features available: iptables, tc, eBPF, ss, conntrack
Full TCP/IP offloadMCU (LwIP or similar)Lower: host sends "open socket / send N bytes" over SPIFewer kernel features; fixed socket count; offload firmware to debug
TX path, end to end
StepWhat happens
app: write()Bytes copied into the socket send buffer
TCP / IPSegmentation, headers, route lookup; output is an skb pointed at wlan0
qdiscPer-device queue: pfifo_fast, fq, fq_codel
ndo_start_xmitSPI driver wraps the skb with a length / type header and queues it
SPI masterDMAs the bytes out on MOSI while reading any pending RX bytes on MISO
MCU firmwareStrips the framing header, hands the payload to its Wi-Fi MAC
802.11 MAC + PHYBuilds the frame, runs CSMA/CA, transmits on the channel

RX is the reverse. The MCU asserts the INT line; the host's GPIO IRQ fires (the INT line is an out-of-band wire, not the SPI controller's own interrupt); the driver does napi_schedule; the NAPI poll then runs spi_sync() in a loop until the MCU says it has nothing more; each frame goes up via netif_receive_skb.

SPI framing

Raw SPI has no concept of packet boundaries: it just clocks bytes. Every co-processor protocol therefore prefixes each transfer with a small header (length, type, sequence, sometimes a CRC). Full-duplex SPI is genuinely full-duplex: while the host clocks out a TX frame, the MCU clocks back any RX it had queued. The INT line is the MCU saying "I have something for you; please drive SCLK".

Skeleton of an SPI netdev driver
/* Match the MCU on the SPI bus (devicetree compatible = "espressif,esp-hosted") */
static const struct of_device_id wifispi_of_match[] = {
    { .compatible = "espressif,esp-hosted" }, { /* sentinel */ }
};

static netdev_tx_t wifispi_xmit(struct sk_buff *skb, struct net_device *ndev)
{
    struct wifispi *priv = netdev_priv(ndev);
    /* Prepend framing: 16-bit length, 8-bit type, 8-bit seq */
    wifispi_frame(priv, skb);
    spi_async(priv->spi, &priv->tx_msg);   /* DMA the bytes; non-blocking */
    return NETDEV_TX_OK;
}

static irqreturn_t wifispi_irq(int irq, void *dev_id)
{
    struct wifispi *priv = dev_id;
    napi_schedule(&priv->napi);             /* MCU has frames to give us */
    return IRQ_HANDLED;
}

static int wifispi_poll(struct napi_struct *napi, int budget)
{
    int n = 0;
    while (n < budget && wifispi_rx_one(napi)) n++;   /* spi_sync inside */
    if (n < budget) { napi_complete_done(napi, n); enable_irq(priv->irq); }
    return n;
}

static int wifispi_probe(struct spi_device *spi)
{
    struct net_device *ndev = alloc_etherdev(sizeof(struct wifispi));
    /* ndo_start_xmit -> wifispi_xmit; ndo_open/stop bring the IRQ up/down */
    netif_napi_add(ndev, &priv->napi, wifispi_poll, NAPI_POLL_WEIGHT);
    return register_netdev(ndev);
}
Bringing it up on the host
CommandWhat it shows
dmesg | grep spiSPI controller probe and the Wi-Fi driver binding to it
ip link show wlan0Link state, MAC, MTU (the SPI link is invisible at this layer)
ethtool -i wlan0Driver name and version (confirms the SPI co-processor driver is in use)
iw dev wlan0 linkSSID, signal, bitrate negotiated by the MCU
cat /proc/interrupts | grep spiSPI IRQ count climbing under load
iperf3 -c hostEnd-to-end throughput; useful for finding the SPI clock or framing bottleneck

Why a board would be built this way: the SBC stays cheap and Wi-Fi-free, the radio module is a pre-certified part you can drop in, and an MCU that was already on the board for sensors can carry the Wi-Fi too. The cost: SPI is the throughput ceiling. A 40 MHz bus delivers tens of Mbit/s once framing and turnaround are paid for, well under what 802.11ac can actually push on air, so this design fits gateways and telemetry, not video.